Method and Apparatus for Controlling Screen by Tracking Head of User Through Camera Module, and Computer-Readable Recording Medium Therefor

ABSTRACT

Controlling a screen by tracking user&#39;s head using a camera module is described, comprising: (a) when a request for displaying contents is received, displaying a plurality of content regions on a screen and tracking the head of the user looking at the screen through the camera module; and (b) when it is determined in the tracking that the head of the user moves in a specific direction in a state where information on a first content is displayed in a central content region among the plurality of content regions, displaying, in the central content region, information on a second content which was displayed in a peripheral content region, the peripheral content region being positioned in the specific direction or in a direction opposite to the specific direction from the central content region.

TECHNICAL FIELD

The present disclosure relates to a method, apparatus, and acomputer-readable recording medium for controlling a screen by trackinga user's head through a camera module. More particularly, the presentdisclosure relates to a method, apparatus, and a computer-readablerecording medium for tracking movement of the head of a user through acamera module and moving a screen according to the movement of theuser's head so that information on a content, which the user wishes toselect from the screen, is displayed in a central content region.

BACKGROUND ART

In a conventional screen controlling apparatus, such as a mobilecommunication terminal, a portable media player (PMP), a television(TV), or a personal computer (PC), an input device, such as a mechanicalkey button, a button type remote controller, or a mouse, has been usedby a user to control a display screen. Recently, due to the developmentof a touch sensing technology, almost all of the functions of an imageoutput device may be controlled, without using the conventional inputdevices, through the user's finger contact with a display panel so thatthe image output device, such as a smart phone, a tablet PC, or a smartTV, may perform more convenient and more various functions than aconventional terminal.

However, in a pressure sensitive touch sensing technology or a resistivetouch sensing technology, touch sensitivity is low. In a capacitivetouch sensing technology, it is difficult to control the image outputdevice when the user wears a glove having different capacitance fromthat of the finger or when the finger is covered with water or a foreignsubstance. In addition, although touch sensing may be used, repetitivefinger contacts with the screen make the user tired and inconvenient.Therefore, a more convenient and simpler input unit for the user isnecessary.

DISCLOSURE OF INVENTION Technical Problem

An object of the present disclosure is to provide a method of trackingthe head of a user through a camera module and controlling a displayedscreen according to the movement of the head of the user to allow theuser to search for and select a desired content from a plurality ofcontents without using any additional input unit.

Further, another object of the present disclosure is to implement a userconvenient interface by allowing various contents to respectivelycorrespond each of three-dimensional grids and by allowing detailedinformation on a specific content positioned in a central content regionto be easily read.

Solution to Problem

The representative configuration of the various embodiments of thepresent disclosure for achieving the aforementioned objects is describedbelow.

According to some aspects of the present disclosure, there is provided amethod of controlling a screen by tracking a user's head through acamera module, comprising: (a) when a request for displaying contents isreceived, displaying a plurality of content regions on a screen andtracking the head of the user looking at the screen through the cameramodule; and (b) when it is determined in the tracking that the head ofthe user moves in a specific direction in a state where information on afirst content is displayed in a central content region among theplurality of content regions, displaying, in the central content region,information on a second content which was displayed in a peripheralcontent region, the peripheral content region being positioned in thespecific direction or in a direction opposite to the specific directionfrom the central content region.

According to another aspect of the present disclosure, there is providedan apparatus for controlling a screen by tracking a head of a userthrough a camera module, comprising: a head tracking unit for trackingthe head of the user looking at the screen through the camera module;and a screen control unit for displaying a plurality of content regionson a screen when a display request of a content is received, and when itis determined in the tracking that the head of the user moves in aspecific direction in a state where information on a first content isdisplayed in a central content region among the plurality of contentregions, displaying, in the central content region, information on asecond content which was displayed in a peripheral content region, theperipheral content region being positioned in the specific direction orin a direction opposite to the specific direction from the centralcontent region.

Advantageous Effects of Invention

According to various embodiments, the present disclosure provides anintuitive and convenient user interface allowing a user to easily accessa specific content corresponding to each grid through simple headmovement, while various contents are mapped to correspondingthree-dimensional grids.

In addition, according to some embodiments of the present disclosure, auser can perform various kinds of instructions/operations for a desiredcontent, including but not limited to selecting, enlarging, reducing,reproducing, and scrolling, through simple head movement.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a view exemplarily illustrating the configuration of a screencontrolling apparatus according to some embodiments.

FIG. 2 is a flowchart illustrating a process of performing a screencontrolling method according to some embodiments.

FIG. 3 is an exemplary view illustrating screens at each stage when ascreen controlling method according to some embodiments is applied.

FIG. 4 is a flowchart illustrating a process of performing a screencontrolling method according to some embodiments.

FIG. 5 is a flowchart illustrating a process of performing a screencontrolling method according to some embodiments.

FIG. 6 is an exemplary view illustrating displayed screens at each stagewhen a screen controlling method according to some embodiments isapplied.

FIG. 7 is an exemplary view illustrating a sequential movement to adetailed screen when a screen controlling method according to someembodiments is applied.

FIG. 8 is a view illustrating a process of controlling a detailed screenwhen a screen controlling method according to some embodiments isapplied.

FIG. 9 is a view illustrating a process of controlling a detailed screenwhen a screen controlling method according to some embodiments isapplied.

FIG. 10 is a view illustrating a process of controlling a detailedscreen when a screen controlling method according to some embodiments isapplied.

BEST MODE FOR CARRYING OUT THE INVENTION

In the following detailed description of the present disclosure,reference is made to the accompanying drawings that show, by way ofillustration, specific embodiments in which various embodiments of thepresent disclosure may be practiced. These embodiments are described insufficient detail to enable those skilled in the art to practice thepresent disclosure. It is to be understood that the various embodimentsof the present disclosure, although different from one another, are notnecessarily mutually exclusive. For example, a particular feature,structure, and characteristic described herein in connection with oneembodiment may be implemented within other embodiments without departingfrom the spirit and scope of the present disclosure. Also, it is to beunderstood that the positions or arrangements of individual elements inthe embodiment may be changed without separating the spirit and scope ofthe present disclosure. The following detailed description is,therefore, not to be taken in a limiting sense, and the scope of thepresent disclosure is defined only by the appended claims along with thefull range of equivalents to which the claims are entitled, whereappropriate.

[Preferred Embodiments of the Present Disclosure]

FIG. 1 is a view exemplarily illustrating the configuration of a screencontrolling apparatus according to embodiments.

Referring to FIG. 1, the screen controlling apparatus may include acamera module 110, a display module 120, a touch sensing module 130, anoperation processing module 160 (including a head tracking unit 140 anda screen control unit 150), and a memory 170.

The camera module 110 may include a lens unit, an image sensor unit, alighting unit, and the like (not shown) in order to photograph an objectand may transfer image sensor information obtained by photographing auser to the operation processing module 160.

The display module 120 receives a display signal from the operationprocessing module 160 to output a screen. The touch sensing module 130may configured to be any one of a resistive touch sensing module, apressure sensitive touch sensing module, and a capacitive touch sensingmodule. The touch sensing module 130 may be combined with the displaymodule 120 and may be exposed separately from the display module 120.The touch sensing module 130 may generate an input signal based on auser contact to transmit it to the operation processing module 160 sothat the operation processing module 160 is allowed to control thescreen to be output through the display module.

The operation processing module 160 includes the head tracking unit 140and the screen control unit 150. The head tracking unit 140 and thescreen control unit 150 are functionally distinguished parts in theoperation processing module 160 both of which operate throughapplication program modules. At least a part of the application programmodules may have a form of an operation system, an application programmodule, and other program modules and may be physically stored in thememory 170 included in the screen controlling apparatus 100 or stored invarious well-known storage devices communicatively connected to thescreen controlling apparatus 100 in a wired or wireless manner. Theapplication program modules may include a routine, a subroutine, aprogram, an object, a component, a data structure, and the like thatperform specific works to be described below or that execute specificabstract data types, but they are not limited to the above.

The head tracking unit 140 recognizes the head of a user or the facethat is a part of the head in an image photographed by the camera module110. When the head of the user or the face that is a part of the head isrecognized, the size, the position, or the angle of the head of the userin the image is calculated in real time. For example, the head of theuser is continuously tracked in the images of 50 frames per second withdetermining whether there is a change in the size, the position, or theangle of the head of the user. The head tracking unit 140 determinesthat a predetermined input is performed through the head of the user ifthere is a change in the size, the position, or the angle of the head ofthe user. In order to remove noises, it may be determined that the userperforms a predetermined input only when the change in the size, theposition, or the angle of the head of the user is continuously kept fora predetermined number of frames, for example, in no less than 10frames, from the frame in which the change in the size, the position, orthe angle of the head of the user is generated. Here, the rotation angleof the head, or the like may be obtained by observing a change infeature points of the face.

However, the head tracking unit 140 is not limited to track the head ofthe user through the change in the size, the position, or the angle ofthe head of the user as described above, but may track the head of theuser by sensing other changes that may be generated in photographedimages according to the movement of the face, such as color changes inthe hair and the face of the user or changes in the shapes of the faceand the neck.

The screen control unit 150 may control the display module 120 todisplay a plurality of content regions when receiving a display requestof content from the user. The display request of the content may bereceived through the touch sensing module 130, and may be input throughan additional mechanical key button or mike not shown.

The screen control unit 150 may display an image photographed by thecamera module as well as the plurality of content regions to overlap apart of the plurality of content regions or in an additional region ofthe screen. In addition, the screen control unit 150 may highlight acentral content region so that the central content region isdistinguishable from other peripheral content regions. For example, thescreen control unit 150 may additionally display a dotted line or acolor line like a viewfinder of a camera in the fixed central contentregion.

The screen control unit 150 controls the display module 120 to display,in the central content region, information on a second content which wasdisplayed in a peripheral region displayed by making the plurality ofcontent regions slide when the head tracking unit 140 determines thatthe head of the user moves upwardly, downwardly or sidewardly in a statewhere information on a first content is displayed in the central contentregion. For example, when the head of the user moves leftwardly, theplurality of content regions slide to the left so that information oncontent which was displayed in a right peripheral content region of thecentral content region may be displayed in the central content region.

Further, the screen control unit 150 may control the screen output fromthe display module 120 based on a combination of whether an additionaltouch input is received from the touch sensing module 130 with a headtracking result. For example, a screen control button may be displayedin a specific region of the screen in a state where the plurality ofcontent regions are displayed. When the screen control button isselected by the user, the screen control unit 150 may allow theplurality of content regions to slide only when there is both themovement of the head of the user and a touch input received through thetouch sensing module 130.

In addition, if the screen control button is selected by continuouslypressing the button for a predetermined time t1, for example, threeseconds, the screen control unit 150 may control the display module 120to prevent the plurality of content regions sliding even though the headtracking unit 140 detects the movement of the head of the user. Afterthen, if the screen control button is selected again by continuouslypressing the button for a predetermined time t2, for example, threeseconds, the screen control unit 150 may allow again the plurality ofcontent regions to slide so that information on content displayed in thecentral content region, for example, a thumbnail image is changedaccording to the movement of the head.

If the information on the specific content is continuously displayed inthe central content region for a predetermined time t3, for example, twoseconds, the screen control unit 150 may control the display module 120to move to a detailed screen for the specific content. Even in thedetailed screen state, the screen control unit may control the detailedscreen to be scrolled, enlarged, or reduced, or to return to the screenof the plurality of content regions, if the head tracking unitdetermines that there is a movement of the head of the user.

The screen displayed as a result of the control as described above willbe described in detail later with reference to FIGS. 3 and 6 to 10.

FIG. 2 is a flowchart illustrating a process of performing a screencontrolling method according to some embodiments.

Referring to FIG. 2, the screen controlling apparatus 100 receives adisplay request of content from a user and displays a plurality ofcontent regions on a screen (S210). The plurality of content regions aregrid-shaped regions including screen units obtained by dividing thescreen region in advance each of which can display information onrespective content. For example, information on images, photos, andmoving images may be displayed in the plurality of content regions,respectively, so that contents such as a photo and a moving imagephotographed by a camera module, an image and a moving image that aredownloaded, and an image and a moving image created by the user may beeasily searched for by the user. Thumbnail images obtained by encodingimages or moving images to images of low resolution may be arranged inthe plurality of content regions so that the user may confirm whichcontents exist at a look. However, the maximum number of thumbnailimages that may be generally displayed on the screen does not exceed atotal of 20, for example, in the case of a grid array of five by four,while the number of contents stored in the memory 170 of the screencontrolling apparatus 100 is larger than 20. Therefore, in order for theuser to search for all of the plurality of stored contents, it isnecessary for the screen showing the plurality of content regions to bemoved in a direction desired by the user.

According to some embodiments of the present disclosure, the screencontrolling apparatus 100 may display information on a first content ina central content region among the plurality of content regions (S220).The central content region as one of the plurality of content regionsmay be positioned, for example, in the center of a screen, and may behighlighted. For example, the frame of the central content region may bedisplayed by a color line or a dot line, the brightness of the inside ofthe central content region is controlled, or a flickering effect isgiven so that the central content region may be distinguished fromperipheral content regions. However, the present disclosure is notlimited to the above.

In this state, according to the present disclosure, the screencontrolling apparatus 100 tracks the head of the user photographed bythe camera module 110 (S230). Head tracking, which tracks the movementof the head of the user, may be implemented by various methods asdescribed above with reference to FIG. 1. According to the presentdisclosure, the movement of the head of the user includes the movementof the face that is a part of the head of the user as described abovewith reference to FIG. 1.

According to some embodiments of the present disclosure, when the screencontrolling apparatus 100 determines that the head of the user moves ina specific direction (S240), information on a second content which wasdisplayed in the peripheral content region of the central content regionis displayed in the central content region, instead of the informationon the first content displayed in the central content region in S220(S250).

For example, when it is determined that the user moves his or her headleftwardly by tilting the head to the left or by rotating the head tothe left in a state where information items on three contents from sideto side (sequentially, a thumbnail image 1, a thumbnail image 2, and athumbnail image 3) are displayed in a left peripheral content region,the central content region, and a right peripheral content region,respectively, the entire screen may slide to the left, for example, sothat the thumbnail image 3 is displayed in the central content region.The central content region may be fixed to the center of the screen tobe displayed.

FIG. 3 is an exemplary view illustrating screens at each stage when ascreen controlling method according to embodiments is applied.

A total of 15 content regions in a grid array of five by three aredisplayed on a screen 300 of the screen controlling apparatus 100illustrated on the left of FIG. 3. In FIG. 3, for the convenience ofexplanation, the plurality of content regions are arranged in the formof a grid. However, the plurality of content regions may be set to beshown to the user as being arranged inside or outside athree-dimensional sphere or cubic and may be set to be shown in the formof a two-dimensional list.

Information items on the respective contents are displayed in theplurality of content regions. The information items on the contents maybe displayed in various forms such as thumbnail images, icons, metadata,text description, and tag information, but are not limited to thespecific data forms and contents. In addition, the contents may be inthe form of audio files, document files, and the like as well as imagesor moving images. In this case, the information items on the contentsmay be displayed as texts that represent the titles, creators, or thelike of documents or audios other than the thumbnail images, or may bedisplayed in the form of images such as icons. In particular, thedocuments may be files having extensions such as doc, docx, hwp, ppt,pptx, xls, xlsx, pps, ppsx, pdf, and gul. In this case, the icons andthe document titles distinguished by the extensions may besimultaneously displayed as the information items on the contents.However, the kinds of the documents or the kinds and forms of theinformation items on the contents are not limited to the above.

Hereinafter, for the convenience of explanation, it is assumed that theplurality of thumbnail images of the images or the moving image contentsare displayed on the screen 300 in the plurality of content regions,respectively.

In the central upper end of the screen 300, an image 310 obtained byphotographing the user by the camera module 110 is displayed in an upperlayer overlapping some of the plurality of content regions. However, theposition of the image 310 obtained by photographing the user is notlimited to the central upper end. The image 310 may be displayed in aseparately assigned region from the plurality of content regions,instead of being layered with the plurality of content regions.

The image 310 obtained by photographing the user may be distinguishedfrom the plurality of content regions by an additional outline, or thelike. When the head or the face 315 of the user is recognized in theimage, a display line for representing three-dimensionally thesilhouette of the head or the eyes, the nose, the lip, and the like ofthe face may be also displayed. The display line may be fixed or maychange to reflect the tracking of the movement of the head in real time.When the display line is fixed, the user may look at the image toconfirm a degree to which the head moves in the image. Meanwhile, whenthe display line is set to move with the head 315 in the image 310obtained by photographing the user, the user may confirm that the headtracking is normally performed.

In addition, for example, the outline of the image 310 obtained byphotographing the user may be displayed to be green when the headtracking is normally performed, while it may be displayed to be red whenthe head or the face cannot be appropriately recognized due to a poorlighting condition such as backlight or the like or a large amount ofnoise information so that the user may confirm the state of the headtracking. Whether the head tracking is normally performed may bepresented by a text message such as “tracking normal” or “tracking fail”or may be provided to the user through an audio output such as a beepersound, other than the above-described method. Such a feedback guides theuser to correct a pose to achieve a lighting condition, a face sizecondition, and a face angle condition in which the head or the faceincluded in the head may be recognized, and allows the user to learn anenvironment, a pose, and a movement appropriate to the head tracking.

When the user moves the head upwardly by tilting the head back or byerecting the neck in a state where a thumbnail image 8 is firstdisplayed in a central content region 320, the screen controllingapparatus 100 determines that the head of the user moves upward andslides the entire plurality of content regions upwardly to allow athumbnail image 13 which was displayed in a peripheral content regionpositioned in the lower side of the central content region 320 to bedisplayed in the central content region 320. Therefore, the user mayposition a desired thumbnail image in the central content region 320only by the movement of the head without using a conventional inputunit. In this case, the arrangement of the information items on theplurality of contents displayed in the plurality of content regions maybe maintained. For example, the entire screen may slide while thearrangement of the thumbnail image 1, the thumbnail image 2, ?, and thethumbnail image 15 is maintained.

When the user moves the head downwardly, according to the similarprocess as described above, the entire plurality of content regionsslide and the thumbnail image 3 which was displayed in a peripheralcontent region positioned in the upper side of the central content imageis displayed in the central content region 320.

Similarly, when the user rotates the head sidewardly or tilts the neckto a shoulder from side to side so that the position of the head itselfis changed, the screen controlling apparatus 100 determines that thehead of the user moves from side to side and slide the entire pluralityof content regions so that the thumbnail image 7 or the thumbnail image9 is displayed in the central content region.

In FIG. 3, the process in which the entire plurality of content regionsslide in the same direction as the movement direction of the head of theuser is illustrated. However, the slide direction of the plurality ofcontent regions is not limited to the above but the entire plurality ofcontent regions may slide in an opposite direction to the movementdirection of the head of the user. For example, when the central contentregion 320 is highlighted by an additional dot line, color line, or thelike which may be recognized as a three-dimensionally or hierarchicallyfloating entity like a viewfinder of a camera, the user may believeintuitively that he or she must move his or her head to the left inorder to select the peripheral content region on the left of the centralcontent region. In this case, the screen may be set to slide in theopposite direction to the movement direction of the head. The screencontrolling apparatus 100 may provide a separate menu for the user toselect whether the slide direction of the screen is set to be the sameas or to be opposite to the movement direction of the head, as desiredby the user.

In the above, a description is made assuming that the plurality ofcontent regions are arranged in the form of a grid. However, even whenthe plurality of content regions are set to be arranged as being insideor outside a polyhedron such as a three-dimensional sphere or a cube,the plurality of content regions may rotate according to the movement ofthe head of the user in a state where the weight center of the sphere orthe cubic is fixed.

FIG. 4 is a flowchart illustrating a process of performing a screencontrolling method according to some embodiments.

The user may not intend to perform the screen controlling methodaccording to the present disclosure whenever he or she moves the head.In particular, if the user searches for information on contents for along time or if the screen controlling method according to the presentdisclosure is performed in a shaking subway or vehicle, an unintendedmovement of the head may be generated. In this case, even though theuser does not desire, a screen slides to change information on contentdisplayed in a central content region.

Referring to FIG. 4, as described in FIG. 2, the head of the user istracked in a state where steps S210 and S220 are performed (S230).

Then, only when it is determined that the head of the user moves in astate where a touch input exists (S400), information on a second contentis displayed in the central content region (S410). In this case, thescreen controlling apparatus 100 may additionally display a screencontrol button on the left and right or in the lower end of the screenso that the user may select the screen control button by touching it.

According to some embodiments, since step S410 is performed only whenthe user touches the screen control button and moves his or her head, itis possible to prevent the screen from sliding due to the unintendedmovement of the head of the user.

In the above, the touch input is assumed in the above description.However, the present disclosure is not limited to the above but it maybe configured to perform step S410 when the movement of the head of theuser is determined and there is an input through an additionalmechanical key button included in the screen controlling apparatus 100or by a sound input such as “up”, “down”, “left”, and “right”.

A screen displayed by applying the screen controlling method illustratedin FIG. 4 will be described in detail later with reference to FIG. 6.

FIG. 5 is a flowchart illustrating a process of performing a screencontrolling method according to some embodiments.

Referring to FIG. 5, the screen controlling apparatus 100 may performsteps S210 and S220 as described in FIG. 2. Then, if the screencontrolling apparatus 100 continuously receives a touch input for apredetermined time t1 (S500), even when it is determined in the trackingthat there is a movement of the head of the user or the face that is apart of the head (S220), unlike in FIG. 1, information on a firstcontent that was displayed in a central content region in step S220 ismaintained in the central content region (S530). For example, if theuser selects a screen control button displayed in a part of a screen bycontinuously pressing it for three seconds, the plurality of contentregions do not slide or rotate so that the information on the firstcontent is displayed in the central content region.

According to another embodiment of the present disclosure, steps S510,S520, and S530 as described above may be replaced by step S501 ofstopping head tracking itself. That is, when the head tracking isstopped, since the plurality of content regions may not slide, the statein which the information on the first content is displayed in thecentral content region is of course maintained.

Although not shown, before performing the above steps S510 and S501, thescreen controlling apparatus 100 may further perform step of displaying,in a region of the screen, a message or a mark representing that thescreen will not be changed although the user moves his or her head. Thatis, a text message such as “Lock” or any other mark may be displayed. Inaddition, a beeper sound meaning that the screen is not changed may beaudio output to inform the user of it.

After then, if an input of continuously contacting the text message orother mark described above for a predetermined time t2, for example,three seconds is received, the image outputting device performs the headtracking and if the movement of the face of the user is detected,performs the step of displaying information on a second content in thecentral content region. That is, a slide function of the screenaccording to the movement of the face is unlocked. Here, t1 and t2 arenot necessarily set to be the same but may be set to vary such that t1is three seconds and t2 is four seconds.

The screen displayed by applying the screen controlling methodillustrated in FIG. 5 will be described in detail below with referenceto FIG. 6.

FIG. 6 is an exemplary view illustrating displayed screens at each stagewhen a screen controlling method according to some embodiments isapplied.

Referring to FIG. 6, a plurality of content regions are displayed on thescreen 300. On the screen 300, the screen 310 obtained by photographingthe face of the user through a camera module is displayed. Informationon a specific content is also displayed in the central content region320. In addition, screen control buttons 600, which are soft buttonsincluding hand shapes illustrated therein, may be displayed on the leftand right of the screen 300. As illustrated in FIG. 4, if the screencontrolling apparatus 100 is set to control the screen to reflect themovement of the head of the user only when an additional touch inputexists, the additional touch input may be received through the screencontrol buttons 600 illustrated in FIG. 6. However, the position, thesize, the shape, and the color of the illustrated screen control buttons600 are merely exemplary and the screen control buttons 600 may bedisplayed in various positions with various shapes.

As illustrated in FIG. 4, although the head of the user moves upwardly,downwardly, or sidewardly, the plurality of content regions do not slideup and down and side to side unless the screen control buttons areselected through the user's finger contact thereto, information on acontent displayed in the central content region is not changed.

On the other hand, if the user moves his or her head while the usercontacts the screen control buttons by a finger or the like, the entireplurality of content regions may slide in the movement direction of thehead so that the information on the content displayed in the centralcontent region is changed. In this case, marks 610 representing that thescreen control buttons are touched may be displayed so that the user mayrecognize that the screen control buttons are selected. In FIG. 6, thescreen control buttons are displayed by the marks 610 each including atext “Touch”, but they may be replaced by various figures or the likethat mean that the screen control buttons are touched.

Meanwhile, if the user continuously touches the screen control buttonfor a predetermined time t1, for example, three seconds, as illustratedin FIG. 5, even though it is determined that the head of the user moves,the plurality of content regions do not slide and the information on thecontent displayed in the central content region is not changed. In thiscase, marks 620 representing that screen control is not performed may bedisplayed. In FIG. 6, the screen control buttons are displayed by themarks 620 each including a text “Lock”, but they may be replaced byvarious figures or the like. Such a state is maintained until the marks620 each including the text “Lock” are pressed again for a predeterminedtime t2, for example, three seconds. However, t1 and t2 are notnecessarily the same but may vary such that t1 is four seconds and t2 isthree seconds.

More specifically, for example, although the user moves his or her headwhile touching the mark 620 including the text “Lock,” the screen doesnot slide. However, when the time for which the user touches the mark620 including the text “Lock” is larger than t2, for example, threeseconds, the screen control buttons 600 are displayed again. Therefore,when the screen controlling apparatus 100 detects the movement of thehead of the user in the state where the screen control buttons 600 areselected, the plurality of content regions may slide so that informationon content displayed in a peripheral content region may be displayed inthe central content region.

FIG. 7 is an exemplary view illustrating a sequential movement to adetailed screen when a screen controlling method according to someembodiments is applied.

Referring to FIG. 7, when, for a predetermined time t3, for example, twoseconds, the thumbnail image 13 of information on a specific content iscontinuously displayed in the central content region 320 with nomovement of the head of the user, the screen controlling apparatus 100makes a movement to a detailed screen in which the original photo orpicture image corresponding to the thumbnail image 13 is displayed indetail.

In this case, it instead of the thumbnail image 13, a loading image 700may be displayed in the central content region 320, which representsthat the screen controlling apparatus 100 is loading the correspondingimage to entirely display it. The loading image 700 may be displayed inthe form of a circular arrow or a bar.

In the above, a description is made assuming that the specific contentis a still image such as an image or a photo and that the information onthe specific content is displayed by the thumbnail image. However, thisis only an example and the specific content may include a moving image,a document, an audio file, and the like, and the information on thespecific content may also be displayed in the form of an icon, a figure,a text, or the like as well as the thumbnail image, as described above.

FIG. 8 is a view illustrating a process of controlling a detailed screenwhen a screen controlling method according to embodiments is applied.

Referring to FIG. 8, it is assumed that a movement has been made to thedetailed screen illustrated in FIG. 7. For the convenience ofdescription, a description will be made with respect to a case in whicha specific content as the Korean flag image is displayed in the detailedscreen.

The image 310 obtained by photographing the user through the cameramodule 110 is output to the center of the upper end of the detailedscreen. Therefore, the user may directly know that his or her head ismoved or tracked.

In the detailed screen, image indicators 800 representing whether aprevious or next content exists are displayed on the left and right ofthe detailed screen. The “previous or next” is based on an arrangementorder of the plurality of content regions displayed before moving to thedetailed screen. For example, in FIG. 6, it may be assumed that athumbnail image (the thumbnail image 12) for the American flag is outputin the left peripheral content region of the central content region 320,that a thumbnail image (the thumbnail image 13) for the Korean flag isoutput in the central content region, and that a thumbnail image (thethumbnail image 14) for the Japanese flag is output in the rightperipheral content region of the central content region. In this case,the “previous” content may be the American flag image and the “next”content may be the Japanese flag image.

The user may determine whether the previous image or the next imageexists through the image indicators 800 without returning to a state inwhich the plurality of content regions are displayed.

In the above state, when it is determined that the user moves his or herhead to the right, the screen controlling apparatus 100 may slide thedetailed screen to the right. Therefore, to the user, the Korean flagimage 810 may be shown as disappearing to the right and the Americanflag image 820 that is a previous image, may be shown as appearing fromthe left. Therefore, according to some embodiments of the presentdisclosure, the screen may be controlled for the user to see theprevious content or the next content in the detailed screen statethrough the movement of the head without returning the display of theplurality of content regions again.

It may be assumed that the user moves his or her head backward in thedetailed screen state so that the head of the user is remote from thecamera module 110 or that the camera module is remote from the user. Inthis case, since the size of the head of the user or the face that is apart of the head of the user is reduced in the image 310 photographed bythe camera module 110, the screen controlling apparatus 100 maydetermine that the head of the user is moved to be remote from thecamera module 100. In this case, the image on the detailed screen, thatis, the Korean flag image is reduced to be displayed.

In addition, when the user moves his or her head to be close to thecamera module or moves the camera module 110 to the face of the user inthe detailed screen state, since the size of the head of the user or theface that is a part of the head of the user is increased in the imagephotographed, the screen controlling apparatus 100 may determine thatthe head of the user is moved to be close to the camera module. In thiscase, the image on the detailed screen, that is, the Korean flag imageis enlarged to be displayed. Therefore, as illustrated in FIG. 8, onlythe taegeuk pattern in the Korean flag image is enlarged to bedisplayed.

In addition, when the screen controlling apparatus 100 determines the upand down and side to side movements of the head of the user in a statewhere the detailed screen is enlarged, the detailed screen may bescrolled while maintaining the state in which the Korean flag image isenlarged. Therefore, the detailed screen, which shows the enlargedtaegeuk pattern, may be scrolled so that the four trigrams of the Koreanfalg, i.e., geon, gon, gam, and ri, may be shown as being enlarged,according to the up and down and side to side movements of the user'shead.

FIG. 9 is a view illustrating a process of controlling a detailed screenwhen a screen controlling method according to embodiments is applied.

FIG. 9 illustrates a state of the detailed screen as described in FIG.7, in which a specific content is a moving image, not an image or aphoto like in FIG. 8.

Referring to FIG. 9, the detailed screen for the moving image may firstdisplay a first frame of the moving image or a screen on which a FIG.900 representing the moving image in the center with a dark coloredscreen.

In this case, when the user tilts his or her head downwardly, the screencontrolling apparatus 100 may determine that the head of the user movesdownward and may reproduce the moving image. When the user nods his orher head downwardly while the moving image is reproduced, the screencontrolling apparatus 100 may determine that the head of the user movesdownwardly and may stop the reproduction and display of the movingimage. Then, the frame screen of the moving image at the time when thereproduction of the moving image is stopped may be displayed as a stillimage. In this case, a predetermined figure representing that thereproduction of the moving image is stopped may also be displayed in thecenter of the stopped frame screen.

When the user moves his or her head upwardly on the detailed screen in astate where the moving image is stopped, it is determined that the headof the user is moved upwardly so that a movement may be made to thescreen on which the plurality of content regions are displayed before amovement to the detailed screen is made.

In the above, it is assumed that the head of the user is moved upwardlyand downwardly. However, the present disclosure is not limited to theabove but it may be controlled such that the reproduction and stop ofthe moving image is also repeated according to the side to side movementof the head of the user.

FIG. 10 is a view illustrating a process of controlling a detailedscreen when a screen controlling method according to some embodiments isapplied.

The image 310 obtained by photographing the user through the cameramodule may be displayed even in a state where a movement to the detailedscreen is made as described above with reference to FIGS. 8 and 9.However, the user may desire that screen control is not performedthrough head tracking on the detailed screen. In addition, since thedetailed screen is a reproduced screen for content, the user may notdesire that a camera screen or the like overlaps the reproduced screen.

Therefore, when the user contacts the image 310 obtained byphotographing the user by the camera module 110 by a finger to generatea touch input, the screen controlling apparatus 100 controls the image310 obtained by photographing the user not to be displayed any further.In this case, the screen controlling apparatus 100 may control thecamera module 110 to hold the photographing of the user and the headtracking.

However, when the user desires again to control the detailed screen bythe movement of the head, the image 310 obtained by photographing theuser through the camera module 110 must be displayed again. Thus, acamera screen output button 950 may be displayed. Therefore, when theuser selects the camera screen output button 950, a photographed imageis displayed again.

The above-described embodiments of the present disclosure can beimplemented as computer readable codes in a computer readable medium.The computer readable recording medium may include but not limited to aprogram instruction, a local data file, a local data structure, or acombination thereof. The computer readable recording medium may bespecific to various embodiments of the present disclosure or commonlyknown to those of ordinary skill in computer software. The computerreadable recording medium includes all types of recordable media inwhich computer readable data are stored. Examples of the computerreadable recording medium include a magnetic medium, such as a harddisk, a floppy disk and a magnetic tape, an optical medium, such as aCD-ROM and a DVD, a magneto-optical medium, such as a floptical disk,and a hardware memory, such as a ROM, a RAM and a flash memory,specifically configured to store and execute program instructions.Examples of the program instruction may include but not limited tomachine code, which is generated by a compiler, and a high levellanguage, which is executed by a computer using an interpreter and soon. The above-described hardware apparatus may be configured to operateas one or more software modules for performing the operation of thepresent disclosure, and the reverse case is similar.

While certain embodiments of the present disclosure have been described,these embodiments have been presented by way of example only, and arenot intended to limit the scope of the disclosures. Indeed, the novelmethods and apparatuses described herein may be embodied in a variety ofother forms; furthermore, various changes, modifications, corrections,and substitutions with regard to the embodiments described herein may bemade without departing from the spirit of the disclosures.

Therefore, the accompanying claims and their equivalents including theforegoing modifications are intended to cover the scope and spirit ofthe disclosures, and are not limited by the present disclosures.

1. A method of controlling a screen by tracking a head of a user througha camera module, comprising the steps of: when a request for displayingcontents is received, displaying a plurality of content regions on ascreen and tracking the head of the user looking at the screen throughthe camera module; and when it is determined in the tracking that thehead of the user moves in a specific direction in a state whereinformation on a first content is displayed in a central content regionamong the plurality of content regions, displaying, in the centralcontent region, information on a second content which was displayed in aperipheral content region, the peripheral content region beingpositioned in the specific direction or in a direction opposite to thespecific direction from the central content region.
 2. The method ofclaim 1, wherein, in the step (a), the plurality of content regions aredisplayed on the screen so that they are arranged in a form of a grid.3. The method of claim 2, wherein the plurality of content regions aredisplayed on the screen so that they are shown to the user to bearranged in a form of a three-dimensional sphere.
 4. (canceled)
 5. Themethod of claim 1, wherein when it is determined in the tracking thatthe head of the user moves in a specific direction in a state where theinformation on the first content is displayed in the central contentregion among the plurality of content regions, displaying, in thecentral content region, the information on the second content which wasdisplayed in a peripheral content region, the peripheral content regionbeing positioned in the specific direction or in a direction opposite tothe specific direction from the central content region, by havinginformation on a plurality of contents displayed in the plurality ofcontent regions slide while keeping their arrangement order.
 6. Themethod of claim 1, further comprising presenting information aboutwhether the head tracking is successfully performed to the user.
 7. Themethod of claim 6, wherein the information about whether the headtracking is successfully performed is presented by changing color of aregion of the screen.
 8. The method of claim 1, wherein the step (b)comprises displaying a predetermined screen control button, and whereinwhen it is determined in the tracking that the head of the user moves ina specific direction in a state where the predetermined screen controlbutton is selected, the information on the second content which wasdisplayed in a peripheral content region is displayed in the centralcontent region, the peripheral content region being positioned in thespecific direction or in a direction opposite to the specific directionfrom the central content region.
 9. The method of claim 1, wherein, inthe step (b), when it is determined in the tracking that the head of theuser moves in a specific direction in a state where a key button signalor an audio signal is input, the information on the second content whichwas displayed in a peripheral content region is displayed in the centralcontent region, the peripheral content region being positioned in thespecific direction or in a direction opposite to the specific directionfrom the central content region.
 10. The method of claim 1, wherein thestep (b) comprises displaying a predetermined screen control button, andwherein if the predetermined screen control button is selected bycontinuously pressing the button for a predetermined time t1, thedisplaying of the information on the second content in the centralcontent region is held until the predetermined screen control button isselected again by continuously pressing the button for a predeterminedtime t2. 11-12. (canceled)
 13. The method of claim 1, further comprisingmaking a movement to a detailed screen for a specific content ifinformation on the specific content is continuously displayed in thecentral content region for a predetermined time without detectingmovement of the head of the user.
 14. The method of claim 13, furthercomprising displaying a loading image for moving to the detailed screenfor the specific content if the information on the specific content iscontinuously displayed in the central content region for thepredetermined time.
 15. The method of claim 13, further comprising, ifit is detected that the head of the user moves in a specific directionin a state where the detailed screen is displayed, making a directmovement to a detailed screen for a previous or next content withreference to the order that the plurality of content regions werearranged on the screen. 16-22. (canceled)
 23. An apparatus forcontrolling a screen by tracking a head of a user through a cameramodule, comprising: a head tracking unit for tracking the head of theuser looking at the screen through the camera module; and a screencontrol unit for displaying a plurality of content regions on a screenwhen a display request of a content is received, and when it isdetermined in the tracking that the head of the user moves in a specificdirection in a state where information on a first content is displayedin a central content region among the plurality of content regions,displaying, in the central content region, information on a secondcontent which was displayed in a peripheral content region, theperipheral content region being positioned in the specific direction orin a direction opposite to the specific direction from the centralcontent region. 24.-33. (canceled)
 34. The apparatus of claim 23,wherein the first content or the second content includes at least one ofphoto, image, moving images, document, and audio contents.
 35. Theapparatus of claim 23, wherein the screen control unit makes a movementto a detailed screen for a specific content if information on thespecific content is continuously displayed in the central content regionfor a predetermined time without detecting movement of the head of user.36. The apparatus of claim 35, wherein the screen control unit displaysa loading image for moving to the detailed screen for the specificcontent if the information on the specific content is continuouslydisplayed in the central content region for the predetermined time. 37.The apparatus of claim 35, wherein, if it is detected that the head ofthe user moves in a specific direction in a state where the detailedscreen is displayed, the screen control unit controls the screen so thata direct movement to a detailed screen for a previous or next content ismade, with reference to the order that the plurality of content regionswere arranged on the screen.
 38. The apparatus of claim 35, wherein, ifit is detected that the head of the user moves in a specific directionin a state where the detailed screen is displayed, the screen controlunit displays an image indicator indicating whether there is a previousor next content, with reference to the order that the plurality ofcontent regions were arranged on the screen.
 39. The apparatus of claim35, wherein, if it is detected that the head of the user approaches thecamera module or is remote from the camera module in a state where thedetailed screen is displayed, the screen control unit allows thedetailed screen to be enlarged or reduced.
 40. The apparatus of claim39, wherein, if it is detected that the head of the user moves in aspecific direction in a state where the detailed screen is enlarged, thescreen control unit allows the detailed screen to be scrolled in thespecific direction or in a direction opposite to the specific direction.41. The apparatus of claim 35, wherein, if the specific content is amoving image and it is detected that the head of the user moves in afirst direction in a state where the detailed screen is displayed, thescreen control unit reproduces the moving image.
 42. The apparatus ofclaim 41, wherein, if it is detected that the head of the user moves inthe first direction while the moving image is reproduced, the screencontrol unit stops the reproduction of the moving image.
 43. Theapparatus of claim 42, wherein, if it is detected that the head of theuser moves in a second direction in a state where reproduction of themoving image is stopped, the screen control unit makes a movement to ascreen on which the plurality of content regions are displayed.
 44. Theapparatus of claim 35, wherein the screen control unit displays a camerascreen captured by the camera module on a part of the screen, andchanges the camera screen to a predetermined screen control button whena touch input is generated on the camera screen.
 45. (canceled)